Initial work to test mongodb-core on evergreen #287

mbroadst · 2018-03-26T14:34:20Z

This includes a number of changes, but generally provides the foundation for testing our two drivers in evergreen, closing: NODE-871, NODE-873, NODE-879, NODE-880, NODE-881, NODE-883, NODE-884, NODE-1191. It also includes a few modifications to the actual tests to reduce flakiness of tests.

You can see it all builds in evergreen here: https://evergreen.mongodb.com/version/5ab7939ce3c331475bc1144a. The red boxes in the result are related to node-snappy not being built to support big endian on those ZAP systems, I have a fix for that here

NODE-884

This just makes our evergreen configuration look a little more like other drivers at very little cost to us. NODE-884

NODE-884

this will pass the option down to `mongodb-download-url` and enable detection of Linux Distributions

This ditches my original custom configuration in favor of one based on the skeleton provided in drivers-evergreen-tools. NODE-884

NODE-884

These tests should either be converted to using the mock server, or we need to update the test configuration to detect whether we are running on evergreen or not, and use the orchestration's rest api to do what we're asking here. NODE-884

NODE-884

…ctory

This is a low-impact change that allows us to potentially make far more accurate assessments of when certain events have completed in the pool. This also allows us to reduce the chances of flakey pool tests

daprahamian · 2018-03-26T15:28:20Z

test/tests/functional/basic_single_server_auth_tests.js

@@ -8,7 +8,8 @@ var expect = require('chai').expect,
  Bson = require('bson');

 describe('Basic single server auth tests', function() {


Is every test skiped in this describe block? We could just do describe.skip

yeah I'm not sure what happened here, I think these were skipped from a while ago but sure I can describe.skip it.

daprahamian · 2018-03-26T15:30:15Z

test/config.js

@@ -2,6 +2,11 @@
 const ConfigurationBase = require('mongodb-test-runner').ConfigurationBase;
 const f = require('util').format;

+const chai = require('chai');


Is there a reason we are adding these things in?

I was actually glad to have found this, we often have cases where we expect(err).to.not.exist, however if it does it shows something like Expected { Object(namespace, message) } to be null which is not helpful at all. The truncation configuration option here expands to show the full message. The other options here are just the defaults, but I'm otherwise codifying that we are indeed using the defaults.

Yeah, this sounds useful. Like I said above though, lets make sure it is a separate commit when we squash.

daprahamian · 2018-03-26T15:30:35Z

lib/connection/pool.js

@@ -210,6 +210,7 @@ function stateTransition(self, newState) {
  // Get current state
  var legalStates = legalTransitions[self.state];
  if (legalStates && legalStates.indexOf(newState) !== -1) {
+    self.emit('stateChanged', self.state, newState);


Is this a bug that needs to be fixed?

I was actually using this to determine when the pool was destroyed, before I opted to use connection spies. It's a harmless addition, but adds flexibility to the design (it's a no-op if nobody is listening) so I'd opt for keeping it in.

Lets make sure that when we squash, this is a separate commit so we can track it.

daprahamian · 2018-03-26T15:34:41Z

.evergreen/run-tests.sh

+download_and_extract "$MONGODB_DOWNLOAD_URL" "$EXTRACT"
+
+# run tests
+OS=$(uname -s | tr '[:upper:]' '[:lower:]')


I feel that things like this could be made into bash functions to make it clearer what is happening.

this is a straight rip of the existing mongo-orchestration scripts, soon to be replaced by work Sam Kleinman is doing with the curator method (read: this is going away soon).

daprahamian · 2018-03-26T15:35:40Z

.evergreen/run-tests.sh

+OS=$(uname -s | tr '[:upper:]' '[:lower:]')
+[ -z "$MARCH" ] && MARCH=$(uname -m | tr '[:upper:]' '[:lower:]')
+
+if [ "$AUTH" != "noauth" ]; then


Can we take strings like this and stick them in variables? I'm worried about things like noauth or nossl being changed in the future.

I think it's a bit premature, there's only a single reference to it. I'll remove the whole section of code actually, since we aren't presently running auth or ssl tests and we aren't likely to use this pattern anyway.

daprahamian · 2018-03-26T15:38:48Z

.evg.yml

+  - id: node-version
+    display_name: "Node.js"
+    values:
+      - id: "Argon"


Are we not making one of these for Carbon?

Yes, Carbon is in the .evergreen/config.yml

daprahamian · 2018-03-26T15:40:07Z

.evg.yml

+        variables:
+           NODE_LTS_NAME: "boron"
+
+buildvariants:


Does evergreen not support >2D matrices? It would be nice to be able to add an axis for Mongo Version

unfortunately not :'(

daprahamian · 2018-03-26T15:42:56Z

.evergreen/config.yml

@@ -0,0 +1,629 @@
+# When a task that used to pass starts to fail


what is the difference between this file and .evg.yml?

ack .evg.yml was accidentally commited. Removing now, and disregarding review from that file.

ok, then transplant all my comments from .evg.yml to evergreen/config.yml

honestly I already squashed a whole bunch of commits here, I'm inclined to leave it as is

mbroadst · 2018-03-26T16:28:04Z

@daprahamian okay comments should be addressed

mbroadst and others added 30 commits March 26, 2018 09:35

fix(test-environments): ensure all servers run on separate ports

dbf4a2f

NODE-884

test(environment): add compat names for mongo orchestration

3e9675e

This just makes our evergreen configuration look a little more like other drivers at very little cost to us. NODE-884

chore(evergreen): add initial evergreen configuration

a780fa6

NODE-884

fix(evergreen): change nvm path to local

3268a89

chore(evergreen): add initial evergreen configuration

21afc38

NODE-884

refactor(evergreen): use type test to reduce system failures

e56a82c

NODE-884

fix(evergreen): change name to id

08c395f

feat(evergreen): use evergreen flag when running tests

8386f0f

this will pass the option down to `mongodb-download-url` and enable detection of Linux Distributions

fix(evergreen): pass in flag through npm scripts

3142eb4

feat(evergreen): add evergreen config based on drivers skeleton

08839c3

This ditches my original custom configuration in favor of one based on the skeleton provided in drivers-evergreen-tools. NODE-884

refactor(test): support environment aliases for evergreen

2bc6ff0

NODE-884

chore(evergreen): ensure variables are passed into run-tests.sh

eebada7

NODE-884

chore(evergreen): add node-version to matrix builds

0d15e6a

chore(evergreen): name environments to property alias for evg

b6b4632

NODE-884

refactor(evergreen): don't show all commands in setup/test scripts

ccdc2d6

NODE-884

test(evergreen): use internal topology manager for tests

f3b2812

test(evergreen): add variables into "run tests" tasks

c467957

test(evergreen): move mongodb download to run-tests

788fff8

test(evergreen): VERSION -> MONGODB_VERSION

e1619a1

test(evergreen): make matrix parity with python driver

6f9ed15

test(evergreen): enable verbose logging for npm install

93ab85f

test(evergreen): store all node related artifacts in a local folder

68fdaeb

test(evergreen): need to re-export NVM_DIR after curl'd install

679b6e4

test(evergreen): ensure all node files are relegated to a single dire…

3ad7edd

…ctory

test(evergreen): use $PROJECT_DIRECTORY for temporary node folder

e8ec942

test(evergreen): temporarily disable auth/ssl builds

322d931

refactor(pool): emit a signal for state changes

e321367

This is a low-impact change that allows us to potentially make far more accurate assessments of when certain events have completed in the pool. This also allows us to reduce the chances of flakey pool tests

test(evergreen): remove suse 11 because its version of git is too old

9bddb5e

test(evergreen): disable windows builds until we have an nvm alternative

ea2f171

mbroadst added 6 commits March 26, 2018 10:32

test(evergreen): only run boron and carbon builds for ZAP architectures

5734e5a

test(config): ensure that error messages are not truncated

3c545ba

test(pool): attempt to correct more pool-related flakey tests

4edfbb5

test(pool): use ConnectionSpy to reduce flakiness of tests

fab8a52

test(cursor): skip the flakiest test in our suite for now

d0e72d5

test(server): don't time how long it takes to eval

873aeba

mbroadst requested a review from daprahamian March 26, 2018 14:34

daprahamian reviewed Mar 26, 2018

View reviewed changes

mbroadst added 3 commits March 26, 2018 12:12

test(evergreen): remove unused bash variables

a3803de

chore(cruft): remove deprecated .evg.yml

129986b

test(auth): skip whole suite rather than individual tests

89dfdbd

mbroadst merged commit 2d5dde3 into 3.0.0 Mar 27, 2018

mbroadst deleted the NODE-884 branch March 27, 2018 11:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial work to test mongodb-core on evergreen #287

Initial work to test mongodb-core on evergreen #287

mbroadst commented Mar 26, 2018 •

edited

Loading

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

daprahamian Mar 26, 2018

mbroadst Mar 26, 2018

mbroadst commented Mar 26, 2018

		@@ -8,7 +8,8 @@ var expect = require('chai').expect,
		Bson = require('bson');

		describe('Basic single server auth tests', function() {

		@@ -0,0 +1,629 @@
		# When a task that used to pass starts to fail

Initial work to test mongodb-core on evergreen #287

Initial work to test mongodb-core on evergreen #287

Conversation

mbroadst commented Mar 26, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbroadst commented Mar 26, 2018

mbroadst commented Mar 26, 2018 •

edited

Loading